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101 TOP LEVEL: 
102 

103 BEGIN 
104 

105 Update database: set the run flag of this daemon to TRUE; 

1 06 Create an empty list V for link storage; 

107 Set boolean Flag to TRUE; 

108 While Flag is TRUE: 



110 Get the user's URL, search interval SI, and depth Depth; 
111 

112 Do searchURL(URL, Depth, 0); 

113 Get the value of the start-stop flag for this daemon from 

114 the database and assign it to Flag; 

115 If Flag is TRUE: 

116 { 

1 1 7 Sleep for SI * 60 seconds; 

118 Get the value of the start-stop flag for this daemon 

119 and assign it to Flag; 

120 } 



121 } 

1 22 Update database: set the run flag of this daemon to FALSE; 
123 

124 END TOP LEVEL 

125 

126 

127 
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128 


SUBROUTINE searchURL(URL, Depth, Level) 


129 






130 


BEGIN 




131 






132 


Create an empty list L for link storage; 


133 


Open two files for writing, an HTML file, and a formatted 


134 


text file (FT file); 




135 


Create a socket connection to the URL; 


136 


Request the HTML from the URL, one character at a time; 


137 


While there is still more data to read from the socket: 


138 


{ 




139 


For each character returned from the URL: 


140 


{ 




141 


Write the character to the HTML file; 


142 


If the character is the beginning of an HTML tag: 


143 


{ 




144 




Read all characters to the end of the current 


145 




tag and store them in a string Tag; 


146 




Write Tag to the HTML file; 


147 




If Tag is a link to another URL: 


148 




{ 


149 




Add the link to list L; 


150 




} 


151 


} 




152 


Else 




153 


{ 




154 






155 




Write the character to the FT file; 


156 


} 




157 


} 




158 


} 




159 






160 


Convert the text in the FT file to one string FS, with each segment of 


161 


white space in FT replaced by a single blank in FS; 


162 
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163 


If the URL is the top-level URL (i.e., Level = 0) 


164 






165 


{ 




166 




Do changeDetection(URL, FS); 


167 






168 






169 






170 




If there is a true change in the text: 


171 




{ 


172 




Set TrueChange to TRUE; 


173 




Add the keyword hit counts generated by 


174 




changeDetectionQ to the database; 


175 




Generate an activity report; 


176 




Add the report to the database; 


177 




Send the report to the user via the user-specified 


178 




notification methods (email, pager, and/or PDA); 


179 




} 


180 




Else 


181 




{ 


182 




Set TrueChange to FALSE; 


183 




} 


184 


} 

> 


185 






186 


Else 




187 


{ 




188 




From the database, get the keywords associated with the 


189 




original top-level URL; 


190 




For each keyword W: 


191 




{ 


192 




Count the number of occurrences N of W in FS; 


193 




Add N to the total keyword count T; 


194 




} 


195 




insen ine crawiea-to ur\i_, trie Keyword oounis in toi edcn 


196 




keyword W, and T into the database; 


1 Q7 

i y / 


} 




198 




199 
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200 


If TrueChange is TRUE OR Level is greater than 0: 


201 








202 


{ 






203 




While List L is not empty: 


204 




{ 




205 




Remove the first URL link U1 from list L; 


206 




If Level is less than Depth: 


207 








208 




{ 




209 




If U1 is not in the list V: 


210 








211 




{ 




212 






If the domain of U1 is identical to the 


213 






domain of the original top-level URL: 


214 






{ 


215 






Insert U1 at the end of list V; 


216 








217 








218 








219 






Do searchURL(U1,Depth,Level+1); 


220 






} 


221 




} 




222 




} 




223 




} 




224 


} 






225 








226 


END SUBROUTINE 
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301 SUBROUTINE changeDetection(URL, FS) 

302 

303 

304 BEGIN 
305 

306 Retreive the formatted text P of the previous version of the URL from 



307 the database; 

308 If at least one character of FS is different from P: 

309 { 

31 0 Replace P in the database with FS; 

31 1 Retrieve the boolean keyword expression Exp associated with 

312 URL from the database; 

31 3 Search the new text FS using Exp; 

314 If Exp is TRUE in FS: 

315 { 

31 6 Get the keywords associated with URL from the database; 

31 7 For each keyword W: 

318 { 

31 9 Count the number of occurrences of W in FS; 

320 } 

321 Retrieve the keyword counts associated with the 

322 previous version P from the database; 



323 
324 
325 
326 
327 
328 
329 
330 
331 
332 
333 
334 

335 Else 

336 { 
337 

338 } 

339 Else 

340 { 

341 No change is detected; 

342 } 
343 

344 END SUBROUTINE 
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If at least one keyword count for FS is different from 
the corresponding keyword count for the previous 
version P: 

{ 

A change between pervious version and new 
version has been detected; 

} 

Else 
{ 

No change is detected; 

} 



No change is detected; 



